Gemma/[Gemma_1]Common_use_cases.ipynb (625 lines of code) (raw):
{
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "Tce3stUlHN0L"
},
"source": [
"##### Copyright 2024 Google LLC."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"cellView": "form",
"id": "tuOe1ymfHZPu"
},
"outputs": [],
"source": [
"# @title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"# you may not use this file except in compliance with the License.\n",
"# You may obtain a copy of the License at\n",
"#\n",
"# https://www.apache.org/licenses/LICENSE-2.0\n",
"#\n",
"# Unless required by applicable law or agreed to in writing, software\n",
"# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"# See the License for the specific language governing permissions and\n",
"# limitations under the License."
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "dfsDR_omdNea"
},
"source": [
"# Gemma - Common use cases\n",
"This notebook demonstrates the basic task that Gemma can solve by using the right prompting.\n",
"<table align=\"left\">\n",
" <td>\n",
" <a target=\"_blank\" href=\"https://colab.research.google.com/github/google-gemini/gemma-cookbook/blob/main/Gemma/[Gemma_1]Common_use_cases.ipynb\"><img src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" />Run in Google Colab</a>\n",
" </td>\n",
"</table>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "FaqZItBdeokU"
},
"source": [
"## Setup\n",
"\n",
"### Select the Colab runtime\n",
"To complete this tutorial, you'll need to have a Colab runtime with sufficient resources to run the Gemma model. In this case, you can use a T4 GPU:\n",
"\n",
"1. In the upper-right of the Colab window, select **▾ (Additional connection options)**.\n",
"2. Select **Change runtime type**.\n",
"3. Under **Hardware accelerator**, select **T4 GPU**.\n",
"\n",
"### Gemma setup\n",
"\n",
"To complete this tutorial, you'll first need to complete the setup instructions at [Gemma setup](https://ai.google.dev/gemma/docs/setup). The Gemma setup instructions show you how to do the following:\n",
"\n",
"* Get access to Gemma on kaggle.com.\n",
"* Select a Colab runtime with sufficient resources to run\n",
" the Gemma 2B model.\n",
"* Generate and configure a Kaggle username and an API key as Colab secrets.\n",
"\n",
"After you've completed the Gemma setup, move on to the next section, where you'll set environment variables for your Colab environment.\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "CY2kGtsyYpHF"
},
"source": [
"### Configure your credentials\n",
"\n",
"Add your Kaggle credentials to the Colab Secrets manager to securely store it.\n",
"\n",
"1. Open your Google Colab notebook and click on the 🔑 Secrets tab in the left panel. <img src=\"https://storage.googleapis.com/generativeai-downloads/images/secrets.jpg\" alt=\"The Secrets tab is found on the left panel.\" width=50%>\n",
"2. Create new secrets: `KAGGLE_USERNAME` and `KAGGLE_KEY`\n",
"3. Copy/paste your username into `KAGGLE_USERNAME`\n",
"3. Copy/paste your key into `KAGGLE_KEY`\n",
"4. Toggle the buttons on the left to allow notebook access to the secrets.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "A9sUQ4WrP-Yr"
},
"outputs": [],
"source": [
"import os\n",
"from google.colab import userdata\n",
"\n",
"# Note: `userdata.get` is a Colab API. If you're not using Colab, set the env\n",
"# vars as appropriate for your system.\n",
"os.environ[\"KAGGLE_USERNAME\"] = userdata.get(\"KAGGLE_USERNAME\")\n",
"os.environ[\"KAGGLE_KEY\"] = userdata.get(\"KAGGLE_KEY\")"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "iwjo5_Uucxkw"
},
"source": [
"### Install dependencies\n",
"Run the cell below to install all the required dependencies."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "r_nXPEsF7UWQ"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m1.1/1.1 MB\u001b[0m \u001b[31m9.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m570.5/570.5 kB\u001b[0m \u001b[31m15.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m311.2/311.2 kB\u001b[0m \u001b[31m3.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m950.8/950.8 kB\u001b[0m \u001b[31m14.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.2/5.2 MB\u001b[0m \u001b[31m17.2 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m589.8/589.8 MB\u001b[0m \u001b[31m2.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.3/5.3 MB\u001b[0m \u001b[31m81.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m2.2/2.2 MB\u001b[0m \u001b[31m95.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[2K \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m5.5/5.5 MB\u001b[0m \u001b[31m85.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
"\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
"tf-keras 2.15.1 requires tensorflow<2.16,>=2.15, but you have tensorflow 2.16.1 which is incompatible.\u001b[0m\u001b[31m\n",
"\u001b[0m"
]
}
],
"source": [
"!pip install -q -U keras keras-nlp"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "pOAEiJmnBE0D"
},
"source": [
"## Exploring prompting capabilities"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "J3sX2mFH4GWk"
},
"source": [
"### Gemma"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "Fz47tAgSKMNH"
},
"source": [
"**About Gemma**\n",
"\n",
"Gemma is a family of lightweight, state-of-the-art open models from Google, built from the same research and technology used to create the Gemini models. They are text-to-text, decoder-only large language models, available in English, with open weights, pre-trained variants, and instruction-tuned variants. Gemma models are well-suited for a variety of text generation tasks, including question answering, summarization, and reasoning. Their relatively small size makes it possible to deploy them in environments with limited resources such as a laptop, desktop or your own cloud infrastructure, democratizing access to state of the art AI models and helping foster innovation for everyone.\n",
"\n",
"**Prompt formatting**\n",
"\n",
"Instruction-tuned (IT) models are trained with a specific formatter that annotates all instruction tuning examples with extra information, both at training and inference time. The formatter has two purposes:\n",
"\n",
"* Indicating roles in a conversation, such as the system, user, or assistant roles.\n",
"* Delineating turns in a conversation, especially in a multi-turn conversation.\n",
"\n",
"Below, we specify the control tokens used by Gemma and their use cases. Note that the control tokens are reserved in and specific to our tokenizer.\n",
"\n",
"* Token to indicate a user turn: `user`\n",
"* Token to indicate a model turn: `model`\n",
"* Token to indicate the beginning of dialogue turn: `<start_of_turn>`\n",
"* Token to indicate the end of dialogue turn: `<end_of_turn>`\n",
"\n",
"Here's the [official documentation](https://ai.google.dev/gemma/docs/formatting) regarding prompting instruction-tuned models."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "B3WckZv2hef3"
},
"outputs": [],
"source": [
"import keras\n",
"import keras_nlp\n",
"from IPython.display import display, Markdown, Latex\n",
"\n",
"os.environ[\"KAGGLE_USERNAME\"] = userdata.get(\"KAGGLE_USERNAME\")\n",
"os.environ[\"KAGGLE_KEY\"] = userdata.get(\"KAGGLE_KEY\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8dfseDZChhjl"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Attaching 'metadata.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'metadata.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'task.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'config.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'metadata.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'metadata.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'config.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'config.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'model.weights.h5' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'metadata.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'metadata.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'preprocessor.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'tokenizer.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'tokenizer.json' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n",
"Attaching 'assets/tokenizer/vocabulary.spm' from model 'keras/gemma/keras/gemma_1.1_instruct_2b_en/3' to your Colab notebook...\n"
]
}
],
"source": [
"# Let's load Gemma using Keras\n",
"gemma_model_id = \"gemma_1.1_instruct_2b_en\"\n",
"gemma = keras_nlp.models.GemmaCausalLM.from_preset(gemma_model_id)"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "N0-VB5iZsVN7"
},
"source": [
"#### Prompting example: Summarization"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "XmavFevGsU-D"
},
"outputs": [
{
"data": {
"text/markdown": [
"Summary:\n",
"Google is a tech giant known for its search engine, email, documents, maps, and more, encompassing various products and services."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"prompt = \"\"\"<start_of_turn>user\n",
"\n",
"Text:\n",
"Tech giant Google, originally a Stanford research project by Larry Page and Sergey Brin,\n",
"has become synonymous with searching the web. Their complex algorithms prioritize relevant\n",
"results, making Google the go-to platform for information seekers. Beyond search,\n",
"Google offers email (Gmail), document creation tools (Google Docs), mapping (Google Maps),\n",
"the Android operating system for smartphones, and the Chrome web browser.\n",
"A leader in AI, Google ventures into self-driving cars and quantum computing.\n",
"Their influence is so vast, \"to google\" has become a verb for searching online.\n",
"\n",
"Could you summarize the text in a few words?<end_of_turn>\n",
"<start_of_turn>model\"\"\"\n",
"\n",
"response = gemma.generate(prompt, max_length=512)\n",
"display(Markdown(response[len(prompt) :]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EidjJx8tBiXd"
},
"source": [
"#### Prompting example: Reasoning"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "8SmUpAFDWjpk"
},
"outputs": [
{
"data": {
"text/markdown": [
"**Step 1: Understand the physics of light scattering.**\n",
"\n",
"- Light is composed of different wavelengths, each with a distinct color.\n",
"- When sunlight enters the atmosphere, it is scattered by molecules and particles in the air.\n",
"\n",
"\n",
"**Step 2: Explain the blue wavelength's scattering effect.**\n",
"\n",
"- Blue light has a shorter wavelength than other colors in the spectrum.\n",
"- When blue light interacts with molecules in the atmosphere, it is scattered more strongly than other colors.\n",
"\n",
"\n",
"**Step 3: Describe the scattering process.**\n",
"\n",
"- Blue light is scattered more efficiently by nitrogen molecules than by oxygen molecules.\n",
"- This means that more blue light reaches our eyes from the sky than other colors.\n",
"\n",
"\n",
"**Step 4: Quantify the effect.**\n",
"\n",
"- The blue sky effect is most pronounced on a clear, sunny day with a high concentration of nitrogen molecules in the atmosphere.\n",
"- Studies have shown that the blue sky effect accounts for about 47% of the visible light that reaches our eyes from the sky.\n",
"\n",
"\n",
"**Step 5: Conclusion.**\n",
"\n",
"- The blue sky is caused by the scattering of blue light by molecules in the atmosphere.\n",
"- This scattering effect is responsible for the blue color we perceive in the sky."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"prompt = \"\"\"<start_of_turn>user\n",
"Think and write your step-by-step reasoning before responding.\n",
"\n",
"Explain why the sky is blue.<end_of_turn>\n",
"<start_of_turn>model\"\"\"\n",
"\n",
"response = gemma.generate(prompt, max_length=1024)\n",
"display(Markdown(response[len(prompt) :]))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "e7P97RVOBkea"
},
"source": [
"#### Prompting example: Classification"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "hCIOmgZhYdow"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sentiment: Positive\n"
]
}
],
"source": [
"prompt = \"\"\"<start_of_turn>user\n",
"Classify the text into neutral, negative, or positive.\n",
"Generate only the class, nothing else.\n",
"Text: Text: I think the food was awesome.<end_of_turn>\n",
"<start_of_turn>model\n",
"Sentiment:\"\"\"\n",
"\n",
"response = gemma.generate(prompt, max_length=64)\n",
"print(\"Sentiment: \" + response[len(prompt) :])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "whC8rU7nBm6O"
},
"source": [
"#### Prompting example: Extraction"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "1YSnZTttZQvT"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Extracted cities: [\"Berlin\", \"Warsaw\", \"London\"]\n"
]
}
],
"source": [
"text = \"\"\"Lost in a worn Berlin travel guide, Ania underlined Warsaw and London.\n",
"Two cities, two dreams: a bustling career in finance and\n",
"a cozy bookshop tucked away in a cobbled London street.\"\"\"\n",
"\n",
"prompt = f\"\"\"<start_of_turn>user\n",
"Your task is to find all the city names in the given text.\n",
"Your response is an array of the model names in the format [\\\"city_name\\\"].\n",
"If you don't find model names in the abstract or you are not sure, return [\\\"\\\"]\n",
"\n",
"Text:\n",
"{text}<end_of_turn>\n",
"<start_of_turn>model\"\"\"\n",
"\n",
"response = gemma.generate(prompt, max_length=512)\n",
"print(\"Extracted cities: \" + response[len(prompt) :])"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "RNpzMyvgBrIr"
},
"source": [
"#### Prompting example: Multi-turn dialogue"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "TZCYYRrXaUYX"
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"\n",
"*** YOU'RE ENTERING CHAT MODE. TYPE 'EXIT' TO FINISH THE CHAT ***\n",
"\n",
"Gemma: Hi there! I'm your friendly AI assistant here to help you with any questions or tasks you may have. I'm here\n",
"to assist you with information, provide solutions, and make your experience more efficient. Let's get started!\n",
"User: Who are you?\n",
"\n",
"Gemma: I am a large language model, trained by Google. I am designed to provide helpful and informative responses\n",
"based on the vast amount of data I have been trained on. I am here to assist you with your questions and requests to the\n",
"best of my abilities.\n",
"User: EXIT\n"
]
}
],
"source": [
"import textwrap\n",
"\n",
"\n",
"def format_messages(messages):\n",
" fmt_messages = [\n",
" f\"<start_of_turn>{author}\\n{text}<end_of_turn>\" for author, text in messages\n",
" ]\n",
" fmt_messages.append(\"<start_of_turn>model\")\n",
" return \"\\n\".join(fmt_messages)\n",
"\n",
"\n",
"history = [\n",
" (\n",
" \"user\",\n",
" \"You're a helpful assistant. Answer user's question in a friendly and helpful way. \"\n",
" \"Please introduce yourself first.\",\n",
" )\n",
"]\n",
"\n",
"print(\"\\n*** YOU'RE ENTERING CHAT MODE. TYPE 'EXIT' TO FINISH THE CHAT ***\\n\")\n",
"while True:\n",
" prompt = format_messages(history)\n",
" response = gemma.generate(prompt, max_length=512)\n",
" response = response[len(prompt) :]\n",
"\n",
" history.append((\"model\", response))\n",
" print(textwrap.fill(f\"Gemma: {response} \\n\", 120))\n",
"\n",
" question = input(\"User: \")\n",
" if question == \"EXIT\":\n",
" break\n",
"\n",
" print()\n",
" history.append((\"user\", question))"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "EOW7M2UABBma"
},
"source": [
"#### Prompting example: Code Generation\n",
"_Note: While Gemma offers some basic code generation capabilities, a more specialized model called CodeGemma is introduced in the next section for these tasks._"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "cqdVbVI14N7B"
},
"outputs": [
{
"data": {
"text/markdown": [
"```python\n",
"def multiply(num1, num2):\n",
" \"\"\"\n",
" This function multiplies two numbers.\n",
"\n",
" Args:\n",
" num1: The first number.\n",
" num2: The second number.\n",
"\n",
" Returns:\n",
" The product of num1 and num2.\n",
" \"\"\"\n",
"\n",
" product = num1 * num2\n",
" return product\n",
"```\n",
"\n",
"**Example Usage:**\n",
"\n",
"```python\n",
"# Multiply 5 and 3\n",
"product = multiply(5, 3)\n",
"\n",
"# Print the product\n",
"print(product) # Output: 15\n",
"```\n",
"\n",
"**Explanation:**\n",
"\n",
"* The function takes two arguments, `num1` and `num2`.\n",
"* Inside the function, we use the multiplication operator (`*`) to calculate the product of `num1` and `num2`.\n",
"* The product is stored in the `product` variable.\n",
"* The function returns the calculated product.\n",
"\n",
"**Additional Notes:**\n",
"\n",
"* This function only works for positive numbers. For negative numbers, a `ZeroDivisionError` will be raised.\n",
"* The function is designed to be simple and efficient. It does not handle errors or special cases."
],
"text/plain": [
"<IPython.core.display.Markdown object>"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"prompt = \"\"\"<start_of_turn>user\n",
"Generate a Python function that multiplies two numbers<end_of_turn>\n",
"<start_of_turn>model\"\"\"\n",
"\n",
"response = gemma.generate(prompt, max_length=1024)\n",
"display(Markdown(response[len(prompt) :]))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"id": "ktdKguk-3T1M"
},
"outputs": [
{
"data": {
"text/plain": [
"{'status': 'ok', 'restart': True}"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# We need to restart the session to release GPU memory\n",
"# before loading different model\n",
"get_ipython().kernel.do_shutdown(True)"
]
}
],
"metadata": {
"accelerator": "GPU",
"colab": {
"name": "[Gemma_1]Common_use_cases.ipynb",
"toc_visible": true
},
"kernelspec": {
"display_name": "Python 3",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 0
}